Embedded system for video processing with hardware means

ABSTRACT

An embedded system for video processing comprises hardware means for realising the following functions:
         i) a server function, wherein for realising the server function the following functions are realised:
           i1) receiving of at least one image-, at least one audio-data stream;   i2) compression of at least one of the image- and audio-data streams yielding in each case a combined data stream respectively;   i3) output of at least one of the combined data streams via a protocol based network;   
           ii) a client function, wherein for realising the client function the following functions are realised:
           ii1) receiving of at least one combined data stream from the protocol based network;   ii2) decompression of the at least one combined data stream;   ii3) output of the combined decompressed image- and audio-data stream;   
           iii) wherein the hardware means are designed in such a way that they compress and decompress a continuous combined data stream and   iv) the hardware means for realising the server function and the client function are arranged on a joint circuit board.

The invention relates to an embedded system for video processing and avideo distribution system. Such devices are particularly used fordistribution of video data from more than one source to more than onedisplay means (i.e. monitors). A typical area of application are bettingagencies. Also further infotainment systems, for which differentinformation or different data streams shall be directed to one or moremonitors for joint output for simultaneous display, for instancedisplays at train stations, airports, in shopping malls etc., are areasof application.

In order to provide TV contents via an IP network, several possibilitiesare known. The distinction between IPTV and internet TV consists of themethod of access. In the case of IPTV, a local network operator specificapplication performs content navigation, whereas in the case of internetTV a standard based browser is used to facilitate the navigation of thecontent. In both cases service video content is provided to a televisionset of a viewer or user via an internet connection.

The EP2247108A1 “Distributed TV access system” discloses an onlineapplication for navigation of broadcasts (broadcast signal, specificallyround or television signals), which are transmitted via a network suchas cable or satellite. A customer receives the broadcast signals via afirst network connection and navigates the televisual contents by meansof an online application via a second network. The first networkconnection and the second network connection may be two connections onthe same network (i.e. two connections via two different software ports)or two connections in two different networks (i.e. the internet and aprivate TV network such as a cable-to-satellite TV network).

The object of the present invention is to provide a system of theabovementioned type, which is versatile. Particularly, by means of theproposed measures a high flexibility in the modes, the connections andresolutions is achieved, and a large variety of combinations in theprocessing of various broadcast signals and the provision of broadcastsignals to the user is accomplished. Furthermore, a preferablescalability concerning the number of participants and the set-up of thesystem is realised and real time capability and security of the signalprocessing as well as the system is enabled.

This object is accomplished according to the present invention by thesubject matter of the independent claims.

The dependent claims constitute preferred embodiments of the invention.

An embedded system for video processing comprising hardware means forrealising the following functions:

-   i) a server function, wherein for realising the server function the    following functions are realised:    -   i1) receiving of at least one image-, at least one audio-data        stream;    -   i2) compression of at least one of the image- and audio-data        streams yielding in each case a combined data stream        respectively;    -   i3) output of at least one of the combined data streams via a        protocol based network;-   ii) a client function, wherein for realising the client function the    following functions are realised:    -   ii1) receiving of at least one combined data stream from the        protocol based network;    -   ii2) decompression of the at least one combined data stream;    -   ii3) output of the combined decompressed image- and audio-data        stream;-   iii) wherein the hardware means are designed in such a way that they    compress and decompress a continuous combined data stream and-   iv) the hardware means for realising the server function and the    client function are arranged on a joint circuit board.

The hardware means are designed in such a way that they can compress ordecompress a continuous image- and audio-data stream with at least 24frames per second, preferably at least 25 frames per second, morepreferably at least 50 frames per second, and most preferably at least60 frames per second, without data jam.

Because the hardware means for realising the server function and theclient function are arranged on a common circuit board, a compactarchitecture is achieved. For user-friendly handling, the joint circuitboard is designed as a PC plug-in card and is displayable in a PCoperating system environment as a network card.

In a preferred embodiment, the system possesses a) hardware means foraccepting more than one combined data stream via the protocol basednetwork;

b) hardware means for decompression of more than one combined datastream;c) hardware means for scaling and/or rotating and/or merging and/orcross-fading of more than one decompressed image and audio data stream;d) hardware means for common output of more than one decompressed imageand audio data stream for simultaneous display on a monitor; ande) hardware means for separate output of one or more combined datastreams on at least two monitors, which are arranged as a video wall.

Thereby, in addition to the usual display of a video data stream to amonitor, it is possible—by scaling means in arbitrary size—to achieve aseries of further display modes, for example two simultaneous streamswith selectable size and positioning can be displayed, or a stream isdisplayed cross-faded over a second one, or four different streams aredisplayed simultaneously, or three streams are displayablesimultaneously on a 90° rotated screen, or six streams are displayableon one monitor (one large, five small), or, for example, a video wallwith four or nine monitors can be configured.

An abundance of information, such as i.e. dynamic live betting odds anddynamic live result services can be displayed in real time, and gamestatistics, parallel TV live broadcasts of several sports events etc.can be clearly edited and displayed, in order to cover the informationcontent on the one hand, and achieve an optimal entertainment experienceon the other hand. Thereby, the streams can be arranged in any order onone or several monitors.

Preferably, the hardware means are designed such that they

-   a. conduct an individual security by means of a hardware serial    number and an individual activation code, and-   b. comprise activation code monitoring means, which conduct a    monitoring and comparison of activation codes.

By means of the activation code and the activation code monitoring meansa relatively high security against product piracy is achieved.Advantageously, the activation code monitoring means are adapted to emitan activation code warning signal or an activation code approval signal.

Preferably, the hardware means are designed in such a way, that they cancompress or decompress a continuous image- and audio-data stream with atleast 60 frames per second and a resolution of at least 1920×1080 pixelswithout data jam. Thus, the embedded system is able to process datastreams in “full HD” resolution. It is self-evident that a resolution of1280×720 is also possible.

Moreover, it is beneficial if the compression of data streams is carriedout according to the H.264- or the H.265-standard. Thereby, anespecially high compression rate for video data streams is accomplished,such that the necessary bandwidth for data transmission via the networkcan be kept as low as possible. The support of other compressionstandards, such as for example MPEG-2, can be provided alternatively oradditionally.

According to a preferable development

-   a. the server function realises a function for encryption of the    combined data stream; and-   b. the client function realises a function for decrypting the at    least one combined data stream.

This encryption and decryption is carried out in real time with hardwaremeans, preferably according to the AES encryption method (“advancedencryption standard”), preferably using a key length of at least 256bit. Thereby the data can be protected from attackers in thetransmission path between server and client.

Furthermore, it is advantageous if the embedded system according to thepresent invention possesses hardware means for converting an image datastream, which has been recorded by interlaced scanning, into progressivescanning. Thereby, it is possible to generate a progressive video datastream in real time and in high quality, even if the video source merelyprovides material, which has been recorded by interlaced scanning(interlaced).

According to a preferable development the embedded system according tothe invention possesses

a) a storage module with a BIOS;b) a storage module with an operating system; as well asc) means for receiving control signals via the protocol based network.

Thereby it is achieved that the embedded system according to theinvention is able to operate independently from other devices(“standalone”), and particularly does not have to be connected to acomputer for operation. However, the embedded system can be controlledvia the protocol based network, for example regarding the choice ofsources of the data streams and the desired transmission format(Server), or for example regarding the choice of the data streams to bedisplayed and the display mode (Client).

A further preferable development of the embedded system according to theinvention possesses means for output of control signals for remotecontrol of connected devices. Thereby, the connected devices, such assatellite television receivers (Server) or monitors (Client), may bealso operated by remote control via the embedded system withoutrequirement of additional remote control units. In doing so, dependingon the kind of device to be operated by remote control, the controlsignals can be outputted via a possibly available infrared interface orvia a possibly available HDMI output or also via the network. Due to thefact that the control signals can be transmitted via the network to theembedded system, it is also possible to operate the connected devicesover longer distances and, if applicable, over structural barriers.Thereby, the operation can be carried out locally for individual or alsocentral for all embedded systems according to the invention connected tothe network. For convenient remote control, the connected devices arerepresentable on a display device, for example a personal computer or atablet computer, and selection menus are allocated to therepresentations in order to select or define display formats, and/or toallocate a specific data stream of a specific server to a client. It isalso possible to display a visualised infrared remote control in orderto adjust monitor specific properties in an easily ascertainable fashionfor the user. The representation of the infrared remote control maycorrespond to a known remote control for the control of a televisionset.

It is also preferable if the embedded system possesses

-   a) hardware means for intermediate storage and repeated output of    the at least one combined data stream; and-   b) means for interactive control of the embedded system by the    observer of the at least one outputted data stream.

It is also preferable if the embedded system possesses hardware meansfor improving the image quality of the image data stream, which providethe following functions: a) changing the brightness and/or b) changingthe contrast and/or c) changing the image sharpness and/or d) noisesuppression and/or e) colour corrections and/or f) reduction ofartefacts and/or g) applying further filters.

Thereby improvements of the image data stream to be displayed can beachieved for the observer in real time. For example, the hardware meansmay be represented on a graphical user interface, and may be controlledby means of allocated selection menus, wherein a user is able towithdraw information about an adjustment from the client, that is thedisplay device, and/or from a representation on the graphical userinterface. Alternatively, the hardware means can be automaticallyadjustable, for example on the basis of available measurement data aswell as comparison data for optimising the actual data of the datastream to be changed.

A further preferred embodiment of the embedded system according to theinvention is characterised in that the hardware means comprise a FPGA(Field Programmable Gate Array), wherein the following functions areprovided by the FPGA: a) connection to the protocol based network; b)firewall; c) encryption of the combined data stream to be output, if theembedded system operates as a server; and d) decryption of the at leastone combined data stream, if the embedded system operates as a client.

Thereby the firewall allows filtering of IP, Port, and Mac addresses viaa rule table. An adjustable logging function of the firewall allowsrecording of valid and/or invalid access attempts on the embeddedsystem. Optionally, unlocked Ports/IPs/Mac addresses can be blockedautomatically after a certain amount of failed attempts or an alarm canbe triggered, if too many not permitted accesses on the embedded systemper unit of time occur.

The FPGA contains in addition a 256 bit AES encryption unit, which isable to encrypt or decrypt the incoming and outgoing network data inreal time. By this realisation of the embedded system a networksecurity, as high as possible, is accomplished.

Furthermore, it is preferable if the configuration of the FPGA isindividually encrypted. Thereby, a security against product piracy isachieved, given that possibly retrieved configuration data of a FPGA ofan embedded system according to the present invention cannot be used forconfiguration of a different embedded system of that kind. To this end,the individual hardware serial number and/or the individual access codecan be used.

A further preferable development of the embedded system according to thepresent invention is characterised in that the hardware means comprise aDSP (digital signal processor), wherein the following functions areprovided by the DSP:

a) compression of at least one of the image- and audio-data streamsyielding in each case a combined data stream, if the embedded systemoperates as a server;b) decompression of the at least one combined data stream, if theembedded system operates as a client; andc) scaling and output of the decompressed image- and audio-data streamvia an interface, if the embedded system operates as a client.

By choosing a suitable DSP an especially high performance during theprocessing of video data is achieved by this realisation of the embeddedsystem, which ensures the real time capability of the embedded system.

It is preferable if the embedded system according to the presentinvention possesses a) at least one network connection, and b) at leastone interface to a data bus and/or c) at least one interface accordingto a video standard, and/or d) at least one interface according to anaudio standard, and/or e) an infrared interface.

The network connection is, in this case, absolutely required, thefurther interfaces or connections, respectively, are optional. Forexample, these may comprise HDMI interfaces, preferably a connection fordata input from a video source and one for data output to a monitor,and/or a PAL interface, and/or an additional stereo audio interface,and/or a USB interface, and/or an infrared interface for operating theconnected video source or the connected monitor by remote control.

It is further favourable, if the protocol based network, to which theembedded system according to the present invention is connected,comprises a transmission capacity of at least 100 Mbit per second.Preferably, the network is a Gigabit-Ethernet-Network, namely a network,which uses the ethernet protocol and has a transmission capacity of atleast 1000 Mbit per second.

Moreover, a video distribution system for receiving and distributing ofat least one video data stream from at least one video source isproposed, with

-   a) a protocol based network;-   b) at least one embedded system described above, which operates as a    server and receiving, compresses and outputs via the protocol based    network at least one of the image data streams from the at least one    video source; and-   c) at least one embedded system described above, which operates as a    client and receives from the protocol based network at least one    compressed data stream, which is decompressed and outputted via an    interface.

Such a video distribution system is, due to its modular set-up and theproperties of the used embedded systems, very versatile and nearlyarbitrary scalable and/or extendible.

It is preferable if regarding the video distribution system according tothe present invention

a) satellite television receivers or cable television connections orDVBT receivers or PCs or media players are deployed as video sources;andb) to the interfaces of the embedded systems, which are operating as aclient, by which the data streams are outputted, monitors are connected.

Furthermore, a video distribution system for receiving and distributingat least one video data stream from at least one video source isproposed, having

-   a) a protocol based network;-   b) at least one embedded system, which operates as a server and    receives, compresses, and outputs via the protocol based network at    least one of the image data streams from the at least one video    source, and-   c) at least one embedded system, which operates as a client and    receives at least one compressed data stream from the protocol based    network, which is decompressed and outputted via an interface.

All the preferable realizations and developments may also be chosen forthe embedded system of said video distribution system, which aredescribed further above for the embedded systems.

A control PC is connected to a network, which comprises image and/oraudio data streams, and a display device for representing at least oneselectable monitor, wherein the control PC comprises at least one inputdevice and, wherein by means of a selection menu, which is displayableon the display device

-   a. the monitor can be switched on or off, and/or-   b. the data streams to be displayed on the monitors are selectable,    and/or-   c. a position of images on at least one monitor is selectable or    adjustable, and/or-   d. a display format is selectable, and/or-   e. a representation of a remote control unit with several switches    is displayable, wherein functions of the monitor are switchable by    means of the switch of the remote control unit.

By means of the graphical user interface all monitors in the system ornetwork can be controlled not only with respect to their adjustment, butalso with respect to the contents to be displayed, by a respectiveallocation of the data streams with the video/audio data, for example TVchannels. Furthermore, a selection and allocation of information data,for example in the form of a so-called news ticker, browser URL, that isinternet pages, and/or image data, is conveniently possible. Obviously,also data streams of several monitors, which are arranged as a videowall, can be assembled individually. By means of the graphical userinterface, standard settings or user settings can be saved and/orloaded. A up-to-date arrangement and assignment of the monitors in aselected network is automatically displayed on the display device.Accordingly, the actually present monitors are visualised and easilyselectable by a user. If a visualised monitor is selected, the user mayopen the allocated selection menu, for example by means of a mouseclick, a keyboard command, or touching a sensor field if the displaydevice is designed as a touch screen. The selection menu displays onlythose menu items (that is control commands), which are executable at therespective monitor or the monitor assembly in the form of the videowall. The control PC may also be a tablet computer or a similar device.In the visualisation, all the adjustments made are displayed, such thatthey are immediately perceivable by the observer. The arrangement ofvisualisation of the available monitors may occur in a user specificfashion or according to a predetermined and saved standard orspecification. During the installation of monitors respective dataregarding the position in a room, their adjustability and addressingand, if applicable, a representation of a specific remote control, aresaved and are subsequently available in the network and, therefore, alsofor visualisation. Concerning a video wall, which also facilitates anoverlaid display of so-called windows with different contents, the videowall as well as the windows are visualised on the display device. Thecontents, that are the data streams, are selectable, and the singlewindows within the video wall are perceivable as images by the user andfreely arrangeable. Each window can be positioned arbitrarily and thearrangement can be saved in a retrievable fashion. Therefore, theobserver of the video wall sees for example a large background image, onwhich two or more further images, also overlapping with each other insections or merging, are arranged. These images, which are visualised aswindows on the display device, may also be scaled in size by means ofthe input device.

Further details and features can be derived from the followingdescription of preferred embodiments in conjunction with the dependentclaims. In that regard the respective features may be realizedindividually or several in combination with each other. Thepossibilities for solving the problem are not restricted to theembodiments. For example, area specifications always comprise all—notexplicitly named—intermediate values and all possible sub-intervals.

It goes without saying that the features named above and to be describedbelow are not only applicable in the respective specified combination,but also in other combinations. The scope of the invention is onlydefined by the claims.

Hereafter, the invention is illustrated further based on examplesreferring to the respective Figures.

It is shown in

FIG. 1 the schematic set-up of an embodiments of the video distributionsystem according to the present invention for application in a bettingagency;

FIG. 2 the set-up of a simple embodiment of the video distributionsystem according to the present invention;

FIG. 3 a more complex embodiment of the video distribution systemaccording to the present invention;

FIG. 4 the schematic set-up of an embedded system according to thepresent invention;

FIG. 5 the schematic set-up of an embedded system according to thepresent invention with a server functionality;

FIG. 6 the schematic set-up of an embedded system according to thepresent invention with a client functionality;

FIG. 7 an illustration of an embedded system according to the presentinvention as a module;

FIG. 8 a schematic representation of the video data processing of anembedded system according to the present invention running as a server;

FIG. 9 a schematic representation of the video data processing of anembedded system according to the present invention operating as aclient; and

FIG. 10 a selection of display modes attainable with an embedded systemaccording to the present invention operating as a client; and

FIG. 11 a software-technological set-up of a branch; and

FIG. 12 a schematic representation of the network security module; and

FIG. 13 a schematic representation of the driver decoupling of aVideoWizard module in a PC operating system environment; and

FIG. 14 an application of the system according to the present inventionas a roulette transmission within a casino; and

FIG. 15 an application of the system as a live game transmission outsideof a casino; and

FIG. 16 a schematic representation of the security of the networktransmission; and

FIG. 17 a schematic overview of the communication structure of thesoftware units, and

FIG. 18 a schematic representation of a user interface of a displaydevice; and

FIG. 19 a schematic partial representation of a single unit XIXaccording to FIG. 19 in an alternative embodiment; and

FIG. 20 a schematic partial representation of a single unit XIXaccording to FIG. 19 in a further alternative embodiment.

The invention comprises a modern video distribution system on the basisof network based multiple H.264 data streams. Thereby, different videosources are converted into H.265 data streams by embedded systemsoperating as a server and decoded by embedded systems operating as aclient and displayed on monitors.

The application of the video distribution system is intended for examplefor betting agencies, where the image sources are normally provided bysatellite receivers and are individually distributed to the available TVmonitors by shop masters/users.

The typical set-up of a video distribution system according to thepresent invention in a betting agency is depicted in FIG. 1 and consistsof the four functional units receiving, distribution, display, andcontrol. The receiving function is realised by a variable number ofsatellite receivers 100, which possess allocated embedded systems 110,which are operating as a server. The encoded video streams of thesatellite receivers 100 are distributed via a network 120 (with aswitch/distributer, VLAN). The display occurs via a variable number ofmonitors 130, which receive their images from allocated embedded systems140, which are operating as a client, wherein the clients 140individually provide from the supply of the receiver servers 110selected image data streams.

Thereby, the selection of the image data streams occurs at local controlPCs 150, at which the shop master/user can dynamically create theallocation between satellite program and monitor 130 (control). To thisend, the TV monitors 130 and the satellite receivers 100 are visualisedgraphically on a display device 155, which is allocated to the controlPC 150, as shown for example in FIG. 18.

Each embedded system 110 or 140, which is connected to a Sat-receiver100 or TV monitor 130, additionally possesses an infrared controlfunction, which allows to remotely operating the allocated device. Everysatellite receiver 100 is also optionally connected to a networkconnection, via which the satellite channel listings can be updated andchanged.

The control of the embedded systems 110 or 140 and, therefore, theallocated devices 100 or 130 occurs physically via network 120 andstructurally via the operator PC software. If necessary, several suchoperator stations 150 allow the control of the devices 100, 130.

According to a preferred embodiment, the system according to the presentinvention constitutes Flexible hardware based multi-function videogaming and visualization system for digital television and digital mediacontent, combination of a freely selectable global hardware based videostream network with game visualisations and interactive live contentdisplay.

According to a preferred embodiment, the system according to the presentinvention is arbitrary scalable by modular set-up (see also FIG. 2 orFIG. 3). A client can access several hundred server streams. The onlylimitation is the capacity of the used network.

According to a preferred embodiment, a division in clients and serversoccurs. Servers receive image data and feed them into the network.Clients receive the network streams and display them in very differentmanners. Interactive control via network (for example PC, IPad, networkcompatible device). Mixed use as client and server simultaneously ispossible.

According to a preferred embodiment, the system according to the presentinvention processes different sources, which can be fed via satellite ornetwork locally or remotely (globally worldwide!) (live cams, digitaland analogue TV sources, hard disk, magnetic disk data, media playeretc.).

According to a preferred embodiment, the system according to the presentinvention is designed for receiving of the image source and real timeencoding according to H.264-methods in hardware (data reduction forsending via network).

According to a preferred embodiment, the system according to the presentinvention is designed of several H.264 source data streams per clientmodule.

According to a preferred embodiment, the system according to the presentinvention is able to provide different visualisation modes by the client(see also FIG. 10): a) 1 stream; b) 2 simultaneous streams; c) 4simultaneous streams per TV; d) 3 rotated streams; e) 6 streams etc.; f)video wall mode, wherein one or several streams can be distributed ontoa wall consisting of many monitors; g) cross-fading mode of differentstreams (blue box); h) Fading/cross-fading of live streams in gameapplication; i) interactive control of the mode by the user; j) replayfunction of video streams on demand.

According to a preferred embodiment, encryption and decryption of datastreams occurs in real time in hardware.

According to a preferred embodiment, the system provides an interactivepossibility to show video contents anew or to change data streams anddisplay modes interactively.

According to a preferred embodiment, transfer via network anddistribution via switch system (i.e. gigabit, Ethernet, UDP protocoletc.) occurs.

According to a preferred embodiment, the system is set-up using combinedFPGA and DSP/special chip technology (real time hardware).

The basic concept of the video distribution system and the embeddedsystems for video processing is depicted schematically in FIG. 2 and isbased on two functional units in combination: the H.264 video functionsfor compression and decompression and the functions for network securitywith hardware firewall and real time data encryption.

H.264 Video Unit

The video unit of the embedded system consists of one or several H.264encoders, which can compress image data in real time using the state ofthe art H.264 method, and one or several H.264-decoders, which candecode and display several compressed image data streams simultaneously.Image and sound data are received via a camera system or a HDMI/PALsource with up to 60 frames per second and transmitted to an embeddedsystem, which is operating as a server, and which processes, records,and compresses these data to a H.264 data format. The image data streamscan then either be transferred uncompressed to a PC or a monitor or canbe transmitted via a network connection. This is depicted schematicallyin FIG. 9.

GigE vision compatible cameras 210, satellite receivers with HDMIoutput, DVD players or other devices adapted for HDMI output serve asimage data source.

The embedded system additionally possesses the opposite functionality(client). It is possible to feed H.264 as well as MPEG2 data streams,which are decompressed in this embedded system in real time and caneither be recoded or directly outputted via HDMI. This is depictedschematically in FIG. 10. Thereby, it is possible to play videos inprogressive and interlaced data format (interlaced scanning) and todeinterlace them as required.

FIG. 10.1 a, 10.2 a shows a single video data stream, which is displayedscaled and positioned on a display (LCD, LED, OLED, TV etc. monitor) andunderlayed with a background colour.

FIG. 10.1 b, 10.2 b shows two single video data streams, which arescaled and positioned and underlayed with a background colour. One ofthe two videos is scaled smaller and is in the foreground.

FIG. 10 c shows two separate video data streams, which are both scaledand positioned and underplayed with a background colour. One of the twovideos is scaled smaller and shown in a blending with the other video(blending).

FIG. 10 d shows four separate video data streams, all four of which arescaled and in a fixed position (non-overlapping).

FIG. 10 e shows three separate video data streams, all three of whichare scaled and rotated by 90°, and in a fixed position(non-overlapping).

FIG. 10 f shows six separate video data streams, all six of which arescaled and have a fixed position (non-overlapping).

FIG. 10 g shows a video data stream on a Video Wall realised with 4monitors, wherein each of the 4 monitors displays a respective sectionof the complete image to be shown.

FIG. 10 h shows three data streams, wherein video data streams as wellas the HTML browser data streams are displayed mixed in differentwindows. The windows are to be arranged scalable, positionable, rotating(rotatable), and overlapping with each other.

FIG. 10 i shows two video data streams in different windows, wherein alarger window displays an image, i.e. from a satellite receiver, and asmaller window shows an information string, for example a so-called newsticker.

The whole system is constructed in such a way that only a single type ofembedded system (VideoWizard module) is required physically. Thisembedded system is designed for security with respect to the datastreams, but also with respect to product piracy.

Thereby, the video distribution system is devised in such a way thatsystems with over 100 servers and clients can be realised in a singlegigabit network stream (this is dependent on the resolution of the datastreams). Alternatively the data stream can be cascaded in a tree likefashion. A system with many components is schematically depicted in FIG.3.

The second simultaneous functionality of the embedded system consists ofa network security functionality, which massively increases datatransmission security via a hardware implemented firewall and AES realtime data encryption. Thereby, the core functions in the present caseare a 256 bit AES encryption of the network traffic as well as ahardware firewall with port activation and logging function.

General Set-Up of the Embedded System:

Conceptually, the embedded system 400 consists of a single component ora module (“VideoWizard hardware”), which is supplemented by amotherboard 710, which contains the additional functions for standaloneoperation.

Functionally, the module is able to realise the server functions, theclient functions, and the network security functions. These aredescribed in more detail below.

By setup, the embedded system (see FIG. 5 and FIG. 8) consists of acombination of FPGA and embedded processor, GPU and special hardwaretechnology (all three functions in the utilised DSP). Thereby, bothunits share different functions. The functionality of the FPGA 410 isnecessary for real time computing and processing of the GigE data of thecamera 210, the processing of HDMI input data, and particularly PCconnection and security aspects, the DSP realises the DVI output andscaling as well as the H.264 data processing as an encoder/decoder.

Advantageously, a driver decoupling of the VideoWizard modules (forexample for different operating systems such as Windows XP Embedded andLinux) is possible by means of the embedded system, wherein theVideoWizard module, which is designed as a PC plugin card, is faded intothe operating system environment as a network card. Thereby, theconnection is a network driver, which allows the use of all standardisednetwork programmes of the operation system. The second access interfaceis realised via a low level user library, which provides the necessarycontrol, image generating and DMA functions. The FPGA can providenetwork functions in combination with the TI co-processor. For example,an image data and control driver 133 is provided in conjunction with aso-called application-programming interface API 134 as well as aseparate network driver 135 (compare FIG. 13 and FIG. 12).

The display device 155, which is allocated to the control PC 150according to FIG. 18, depicts a graphical user interface forvisualisation of the available monitors 156, and/or windows 157, and/ora remote control unit 158. By means of selection menus 154 the monitors156 can be switched on or off, the data streams to be displayed on theon monitors 156 or in the windows 157, which represent screen sections,can be selected, a display format can be selected, and the display sizecan be defined. The data streams to be selected individually maycomprise video data, information data, for example in the form of aso-called news ticker, browser URL, that is Internet pages, and/or imagedata. By means of the graphical user interface it is also possible tosave and/or load standard settings and/or user settings.

According to FIG. 19 a large-scale image A′, which is visualised on thedisplay device 155, is displayed on several monitors, which areassembled to a video wall 159, which is partially overlapping with twowindows 157, which display the images B′ and C′, wherein the window 157with the image C′ partially overlaps with the window 157 with the imageB′. In order to make adjustments concerning the window 157 with theimage C′, the window 157 is selected by means of an input device, forexample a mouse, which is allocated to the control PC 150 and thedisplay device 155, as depicted by the grey background, and theselection menu 154, which is allocated to the window 157, with differentmenu items 153, is expanded visually perceptible. After the selection ofthe menu item 153, depicted in grey, the allocated remote control unit158 appears, with whose switches adjustments are to be made, or sent tothe respective monitors or clients by means of especially adjustedcommands. Obviously, the display device 155 may be designed as a touchscreen, whereby the operation by a user can be very easily realised.

The representation of configurations, particularly the visualisedassembly and sizes of the monitors 156 or windows 157 on the displaydevice 155, corresponds to the actual assembly of monitors or is definedby means of a specified image. Previously saved configurations can beselected under a respective menu item 153, which can also be displayedin a title bar on the display device 155. By means of the input device,a selection menu with the representation of all available clients can bedisplayed.

The following types of contents can be defined in the visualisation ofthe display device 155:

-   -   image: An image defined by the user is displayed. Images stored        on an image server in a specific directory can be selected.    -   info display: All available channels in the info display system        can be selected.    -   TV channels: Modules recognised as a server can be selected as        TV channels.    -   Browser URL: Any URL can be selected in this menu, however        browser support on the modules can be restricted, i.e. no flash        contents displayed.

With the menu item 153 “distribution” in XML format defined layouts maybe allocated, which are available dynamically, meaning that possiblelayouts are only displayed for the present monitor or group of monitors.

Description of a XML Configuration:

Layout <Layout id=“xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx”xmlns=“urn:admiral:absolutevision:light:layout:v1”>

xxx . . . corresponds to a GUID, which is unique in the directory Thefile name of the configuration should also comply with this GUID.

GUID can be created by a GUID generator. i.e.:http://www.guidgenerator.com/online-guid-generator.aspx

GUID (or UUID) is an acronym for “Globally Unique Identifier” (or“Universal Unique Identifier). It can be a 128-Bit integer foridentifying resources.

The name entry is used by the system, therefore a name, which describesthe distribution, should be entered.

<Name>Split 2/PictureInPicture</Name>

Under translations the name of the distribution is entered, which isdisplayed in the GUI. The translation according to the Windows systemlanguage is displayed. If there is no entry for the selected systemlanguage, the translation en is displayed.

<Translations>

<Translation language=“de”>2 Anzeigen/Bild in Bild</Translation>

<Translation language=“en”>2 Views/Picture in Picture</Translation>

</Translations>

<Screens> describes how the monitors or screens are arranged in thecoordinate system, their resolution, and the rotation.

<Screens>

<Screen x=“0” y=“0” width=“1920” height=“1080” rotation=“0”></Screen>

<Screen x=“0” y=“1080” width=“1920” height=“1080” rotation=“0”></Screen>

</Screens>

Thereby, all monitors or screens, which are comprised in thedistribution, should be entered. x=“0” y=“0” describe the position ofthe monitor/screen in the coordinate system of the video wall. Thereby,for instance the upper left point of the display can be used as areference for positioning.

For example width=“1920” height=“1080” describe the size of the display.

rotation=“0” describes the orientation, namely rotation of the display.For instance, only the values 0°, 90°, 180° und 270° can be supported.Currently, only 0 und 90 make sense, because only horizontal andvertical orientations can be selected in device settings.

<Contents> defines the viewing areas in the coordinate system. Theseareas can span several monitors.

<Contents>

<Content x=“0” y=“0” width=“1920” height=“1080”></Content>

<Content x=“200” y=“200” width=“640” height=“480”></Content>

</Contents>

x=“0” y=“0” correspond to the starting coordinates of the content areain the coordinate system. For example, the upper left point can be usedas a reference for positioning

width=“1920” height=“1080” describe the size of the viewing area. If acontent, which does not correspond to the indicated aspect ratio, isallocated to the content window, the content is adapted to the contentarea and scaled.

Example according to FIG. 20:

A video wall 159 consists of six FullHD TVs (3×2) and three viewingareas E′, F′, G′:

Viewing area E′: Content over the whole wall of 3×2 TVs

Viewing area F′: Content over a partial area of 4 TVs. The displayoverlaps with viewing area E′

Viewing area G′: Content over a partial area of 4 TVs. The displayoverlaps with viewing area E′ and a part of viewing area F.

An accordingly constructed XML configuration file can for instance betransmitted to the client modules via an API interface.

Server Functionality

The server functionality of the embedded system is depictedschematically in FIG. 5 and partially in FIG. 12 (left of the network)and consists of the following detail functions:

-   -   receiving of HDMI data streams up to 1920×1080 p60 (progressive,        60 Hz) or camera data streams;    -   compression of the image- and audio-data streams in H.264        format;    -   packaging of control and audio data into the H.264 data stream;    -   standalone capability with access (via server PC 510) via        Gigabit-Ethernet to internal registers of the embedded system;    -   individual FPGA security against product piracy;    -   AES encryption of the data stream;    -   sending the compressed data via GigE network

Client Functionality

The client functionality of the embedded system is depictedschematically in FIG. 6 as well as partially in FIG. 12 (right of thenetwork) and consists of the following detail functions:

-   -   receiving of the compressed data or a data stream, respectively,        via GigE network;    -   AES decryption of the data stream;    -   decompression of the image- and audio-data from the H.264 format        or extraction of the control data from the data stream,        respectively;    -   decompression of the MPEG2 image data streams where applicable;    -   deinterlace function with high quality for realisation of        progressive image material;    -   image scaling;    -   restoration of an image data stream with 60 frames per second        from the H.264 data streams;    -   standalone capability with access (via client PC 610) via        Gigabit-Ethernet to internal registers of the embedded system;    -   individual FPGA security against product piracy;    -   output of video audio and control data;

The embedded system (the VideoWizard) can preferably be utilised incombined modes. For example, a mix of client and server (with aselection of partial functions) or a recoding of several input datastream to new output streams is possible.

Network Function (Firewall and Data Encryption)

The network functionality of the embedded system consists of a hardwareimplemented firewall and AES real time data encryption, whereby the datatransmission security is massively increased. Thereby, the corefunctions in the present case are a 256 bit AES encryption of thenetwork traffic and a hardware firewall 131 with port activation andlogging function. In detail the FPGA 410 predominantly realises thenetwork functions (FIG. 13). Thereby, the FPGA contains a 256 bit AESreal time encryption unit 132, which encrypts or decrypts, respectively,the incoming and outgoing network data. It is possible to choose betweendifferent H.264 data streams, which are transmitted into/out of theembedded system via network. The firewall 131 allows filtering of IP,port, and Mac addresses via a rule table. An adjustable logging functionof the firewall 131 allows recording of the valid/invalid accessattempts onto the embedded system. Optionally unlocked ports/IPs/Macaddresses can be blocked automatically after a certain amount of failedattempts or an alarm can be triggered if too many unallowed accesses tothe embedded system occur per second. Regarding this matter, failedattempt monitoring means, which monitor the occurrence of failedattempts exceeding a defined threshold, are provided in the system.

Security Against Product Piracy

By means of an individual hardware serial number saved in the embeddedsystem, it can be ensured that the embedded system can only be operatedwith an individual activation code. Therefore, a copy of the module forduplication is pointless. In the system, activation code monitoringmeans 161 can be provided, which are able to perform a monitoring orcomparison of activation codes with a database and, if necessary, canemit an activation code warning signal if no match of the activationcode or codes occurs or can emit an activation code approval signal if amatch of the activation code or codes occurs.

Security During Image and Data Transmission

Optionally, the image data streams can be secured via the network withan AES encryption method, whereby the data is protected from an attackerin the transmission path. This concerns data transmission via Internetas well as internal transmission, for example in a casino.

High stability and safeguard against failure during image datatransmission and display. By the utilisation of two independent realtime capable computing components a high operating safety with highoutput reserves is realised.

Stability by Independence

By means of the independence of the embedded system from a PC control,the stability of the overall application is increased. In case of afailure of a part of the overall system, the embedded systems continueto function independently.

Network Security by a Hardware Firewall

By use of a hardware firewall the unauthorised access from outside vianot permitted ports, not activated IP addresses, or not authorised Macaddresses can be prevented. Moreover, the logging allows a recording ofpermitted and non-permitted accesses. By utilisation of a hardwarefirewall, the firewall cannot let accidentally, in case of capacityoverload, not permitted packages pass, as is the case forsoftware-implemented firewalls.

Numerous modifications and developments of the described embodiments arerealisable. For instance, an embedded system according to the presentinvention can also be used without a network, wherein it provides theserver and client functionality simultaneously. Furthermore, differentcoding and encryption standards or methods can be utilised, and alsoaccording to the desired area of application, different interfaces,network protocols and network architectures can be used without changingthe essence of the invention.

GLOSSARY

Image data stream, audio data stream, video data stream, combined datastream We consider these terms to mean data streams with a respectivechannel of the respective content (images, audio, video). A combineddata stream contains image as well as audio data. Therefore, a videodata stream can be an image data stream or a combined data stream.

Client/Server

We consider the terms client and server to mean so-called streamingclients or streaming servers. The term streaming client designates aspecial client for streaming media, which can be either software orhardware. Typical streaming clients support special streaming protocolssuch as RTP, RTCP, and/or RSVP. The term streaming server designates adecided server for distribution of streaming media data via a network.Typical streaming servers support special streaming protocols such asRTP, RTSP, RTCP, and RSVP (auxiliary protocol for QoS-method IntServ).Thereby, the term streaming media designates from a computer networkreceived and simultaneously played audio and video data. The process ofdata transmission itself is termed streaming, and transmitted(“streamed”) programmes are termed live stream or shortly stream.

(According to http://de.wikipedia.org/wiki/Streaming-Client,http://de.wikipedia.org/wiki/Streaming-Server andhttp://de.wikipedia.org/wiki/Streaming_Media)

Embedded System

The term embedded system (Eng. embedded system) describes an electronicdata processor or computer, which is embedded (embedded) into atechnical context. Thereby, the data processor assumes eithermonitoring, control, or regulatory functions or is responsible for aform of data or signal processing, for example during encryption ordecryption, encoding or decoding, or filtering.

(Cited according to http://de.wikipedia.org/wiki/Eingebettetes_System)

Hardware Means

We consider the term hardware means to mean electronic components, whichare optimised for certain purposes and execute their function mainly inreal time. Typically, DSP, programmable hardware such as FPGAs, specialASICs (application-specific integrated circuit), or combinations thereofare used for such purposes. These can also contain embeddedmicroprocessors (i.e. ARM processors). In the context of thisapplication the term hardware means does not comprise multi-purposemicroprocessors, which are not optimised for the purposes named herein.Multi-purpose microprocessors (i.e. by Intel or AMD) are typically foundin personal computer as central processing units.

DSP

A digital signal processor or DSP serves the purpose of continuousprocessing of digital signals (i.e. audio or video signals). Forprocessing of analogue signals, the DSP is used in conjunction withanalogue-digital converters and digital-analogue converters. Compared tomulti-purpose microprocessors DSPs contain a processor, which isspeed-optimised for frequently required mathematical operations. SomeDSPs already contain the necessary A/D and D/A converters at their inputand output.

(Cited according tohttp://de.wikipedia.org/wiki/Digitaler_Signalprozessor)

A field programmable gate array or FPGA is an integrated circuit, intowhich a logic circuit can be programmed.

However, the term programming in this context is to be distinguishedfrom the term describing the design of software for a conventionalprocessor: In a FPGA circuit structures are generated by means ofhardware description languages or in the form of wiring diagrams andthese data are transmitted into the component for configuration.Thereby, certain switch positions are activated or deactivated,resulting in a concretely implemented digital circuit. Therefore,instead of the term programming the term configuration of a FPGA is alsoused. By the specific configuration of internal structures differentcircuits can be realised in a FPGA, culminating in highly complexstructures, such as i.e. microprocessors.

The configuration typically takes place once before each use, wherebythe FPGA is configured for a specific function, which it loses againupon switching off of the operating voltage. Therefore, a non-volatilememory, which stores the configuration, whose content on itself is alsoupdatable, is allocated to the FPGA in most cases.

(according to http://de.wikipedia.org/wiki/Fpga)

Ethernet, GigE

Ethernet is a technology, which specifies software (protocols etc.) andhardware (cables, splitters, network cards etc.) for tethered datanetworks. It facilitates data exchange in the form of data packagesbetween the devices connected to a local network (LAN).

Firewall

A firewall is a security system, which protects a network or a singlecomputer from unwanted network accesses. The firewall serves to restrictthe network access based on sender or destination address and usedservices. It monitors data traffic and decides according to specifiedrules, where certain network packages can pass through or not. Thesimple filtering of data packages according to network addresses is thebasic function of all firewalls.

HD

This abbreviation stands for “high definition”, (Eng.) wherein in ourcase image resolutions of 1280×720 or 1920×1080 pixels are meant,wherein the higher resolution is also termed “full HD”.

Video resolution comprises the same parameters as image resolution(lines and columns or pixel number, aspect ratio) and extends these bythe temporal aspect of frame rate. Thereby, it has to be differentiatedbetween the repetition of partial (mostly half images with interlacedscanning, interlaced), and full images (frames, progressive scan).Common frame repetition rates are 24, 25, 50, or 60 Hz. In the HDTV area720p and 1080i are common. One speaks of full HD from 1080p25, whichmeans 1920×1080 pixels, progressive, 25 Hz. (According tohttp://de.wikipedia.org/wiki/Videoauflösung,http://de.wikipedia.org/wiki/Full_HD andhttp://de.wikipedia.org/wiki/High_Definition_Television)

Video

We consider video to mean moved images or image streams, with or withoutsound.

Interlaced Scanning, Progressive Scan

Interlaced scanning serves to reduce flicker in television engineering.It was developed with the intention to display signals flicker free witha minimal bandwidth. Thereby, a complete image (frame) is constructedfrom two different half images.

The progressive scan (Eng. progressive scan) designates a technique inimage construction, wherein the output device—in contrast to interlacedscanning—does not receive line interlaced half images, but real fullimages.

(Cited according to http://de.wikipedia.org/wiki/Zeilensprungverfahrenand http://de.wikipedia.org/wiki/Vollbildverfahren)

1. An embedded system for video processing with hardware means forrealising the following functions: a) a server function, wherein forrealising the server function the following functions are realised: a1)receiving of at least one image-, at least one audio-data stream; a2)compression of at least one of the image- and audio-data streamsyielding in each case a combined data stream; a3) output of at least oneof the combined data streams via a protocol based network; b) a clientfunction, wherein for realising the client function the followingfunctions are realised: b1) receiving of at least one combined datastream from the protocol based network; b2) decompression of the atleast one combined data stream; b3) output of the combined decompressedimage- and audio-data stream; c) wherein the hardware means are designedin such a way that they compress and decompress a continuous combineddata stream; and d) the hardware means for realising the server functionand the client function are arranged on a joint circuit board.
 2. Theembedded system according to claim 1 characterised in that the jointcircuit board is designed as a PC plug-in card and is displayable in aPC operating system environment as a network card.
 3. The embeddedsystem according to claim 1 characterised by a) hardware means foraccepting more than one combined data stream via the protocol basednetwork; b) hardware means for decompression of more than one combineddata stream; c) hardware means for scaling and/or rotating and/ormerging and/or cross-fading of more than one decompressed image andaudio data stream; d) hardware means for joint output of more than onedecompressed image and audio data stream for simultaneous display on amonitor; and e) hardware means for separate output of one or morecombined data streams on at least two monitors, which are arranged as avideo wall.
 4. The embedded system according to claim 1 characterised inthat the hardware means are designed in such a way that they a. conductan individual security by means of a hardware serial number and anindividual activation code, and b. comprise activation code monitoringmeans, which conduct a monitoring and comparison of activation codes. 5.The embedded system according to claim 1 characterised in that theactivation code monitoring means are adapted to emit an activation codewarning signal or an activation code approval signal.
 6. The embeddedsystem according to claim 1 characterised in that the hardware means areadapted in such a way that they compress and decompress a continuousimage- and audio-data stream with at least 60 frames per second and aresolution of at least 1920×1080 pixels without data jam.
 7. Theembedded system according to claim 1 characterised in that thecompression of the data streams is carried out according to the H.264 orthe H.265 standard.
 8. The embedded system according to claim 1characterised in that c. the server function realises a function forencrypting the combined data stream; and d. the client function realisesa function for decrypting the at least one combined data stream.
 9. Theembedded system according to claim 1 characterised by hardware means forconverting an image data stream, which has been recorded by interlacedscanning, into progressive scanning.
 10. The embedded system accordingto claim 1 characterised by a) a storage module with a BIOS; b) astorage module with an operating system; and c) means for receivingcontrol signals via the protocol based network.
 11. The embedded systemaccording to claim 1 characterised by means for output of controlsignals for remote control of connected devices.
 12. The embedded systemaccording to claim 1 characterised by a) hardware means for intermediatestorage and repeated output of the at least one combined data stream;and b) means for interactive control of the embedded system by theobserver of the at least one outputted data stream.
 13. The embeddedsystem according to claim 1 characterised in that the hardware meanscomprise a FPGA, wherein the following functions are provided by theFPGA: a) connection to the protocol based network; b) firewall; c)encryption of the combined data stream to be output, if the embeddedsystem operates as a server; and d) decryption of the at least onecombined data stream, if the embedded system operates as a client. 14.The embedded system according to claim 1 characterised in that theconfiguration of the FPGA is individually encrypted.
 15. The embeddedsystem according to claim 1 characterised in that the hardware meanscomprise a DSP, wherein the following functions are provided by means ofthe DSP: a) compression of at least one of the image- and audio-datastreams yielding in each case a combined data stream, if the embeddedsystem operates as a server; b) decompression of the at least onecombined data stream, if the embedded system operates as a client; andc) scaling and output of the decompressed image- and audio-data streamvia an interface, if the embedded system operates as a client.
 16. Theembedded system according to claim 1 characterised by a) at least onenetwork connection, and b) at least one interface according to a videostandard, and/or c) at least one interface according to an audiostandard, and/or d) at least one interface to a data bus and/or e) aninfrared interface.
 17. A video distribution system for receiving anddistributing of at least one video data stream from at least one videosource, with a) a protocol based network; b) at least one embeddedsystem according to one of the preceding claims, which operates as aserver and receiving, compresses and outputs via the protocol basednetwork at least one of the image data streams from the at least onevideo source; and c) at least one embedded system according to one ofthe preceding claims, which operates as a client and receives from theprotocol based network at least one compressed data stream, which isdecompressed and outputted via an interface.
 18. The video distributionsystem according to the preceding claim claim 17, wherein a) satellitetelevision receivers, or cable television connections, or DVBTreceivers, or PCs or media players are used as video sources; and b) tothe interfaces of the embedded systems, which are operating as a client,by which the data streams are outputted, monitors are connected.
 19. Acontrol PC, which is connected to a network, which comprises imageand/or audio data streams, and a display device (155) for representingat least one selectable monitor, wherein the control PC (150) comprisesat least one input device and, wherein by means of a selection menu(154), which is displayable on the display device (155) a. the monitorcan be switched on or off, and/or b. the data streams to be displayed onthe monitors are selectable, and/or c. a position of images on at leastone monitor is selectable or adjustable, and/or d. a display format isselectable, and/or e. a representation of a remote control unit (158)with several switches is displayable, wherein functions of the monitorare switchable by means of the switch of the remote control until (158).20. The control PC according to claim 19 characterised in that theremote control unit (158) is displayable by selection of a selectionmenu (154), which is assigned to a selected monitor.
 21. The control PCaccording to claim 20 characterised in that the display device (155)represents the arrangement of monitors, and/or their content accordingto an up-to-date configuration, wherein data of the configuration areavailable in the network.
 22. The control PC according to claim 19characterised in that by means of the input device the images on atleast one monitor are to be arranged in an overlapping fashion on thedisplay device (155).
 23. The control PC to claim 19 characterised inthat an embedded system according to one of the preceding claims and/ora video distribution system according to one of the preceding claims isconnected to the network.